Skip to content

fix(runner): verify the datadir head for a bundle inside the fixtures artifact - #306

Merged
skylenet merged 1 commit into
masterfrom
fix/prerun-head-verify-artifact-bundle
Aug 11, 2026
Merged

fix(runner): verify the datadir head for a bundle inside the fixtures artifact#306
skylenet merged 1 commit into
masterfrom
fix/prerun-head-verify-artifact-bundle

Conversation

@skylenet

Copy link
Copy Markdown
Member

Fixes #305.

verifyPreRunBundleHead exists to guard the replay's by-number skip — a datadir at the bundle's height but on a different chain would otherwise skip every line and benchmark the wrong state. It never ran for artifact-resolved bundles:

if src == nil || src.PreRuns == nil || src.PreRuns.LocalFixturesDir == "" {
    return false, nil
}

Since #296 a pre_runs source can resolve its bundle out of the extracted fixtures artifact using only fixtures_subdir, leaving local_fixtures_dir empty — so the guard returned early for exactly the configs that depend on that path. That gap came in with #296, which taught loadPreRunBundleSteps the fallback and left this second call site behind.

Change

Ask the source where its bundle is instead of re-deriving the path from config. The bundle may sit at a configured local directory or inside the extracted artifact, and only the source knows which — a second config-derived derivation is what silently missed the artifact case.

// optional, alongside the existing GenesisProvider idiom
type PreRunBundleLocator interface {
    PreRunBundleDir() string
}
  • EESTSource.PreRunBundleDir() resolves it once and is now the single source of truth; loadPreRunBundleSteps uses it too, so the replay and the check can no longer disagree about which bundle is in play.
  • Implemented as an optional interface checked by type assertion, so LocalSource, GitSource and ArchiveSource are untouched.
  • ReadPreRunBundleInfoAt(bundleDir) reads a directory named outright, which is what a runner-side pre_runs source has. ReadPreRunBundleInfo(parent) keeps its parent + PreRunBundleSubdir form and simply delegates.
  • The config-derived path stays as a fallback for callers without a live source.
  • An unlocatable bundle now warns rather than passing quietly — the skip it guards runs either way, so silence there is the dangerous default.

No behaviour change for a local_fixtures_dir config: it resolves to the same directory by either route.

Why it matters

Two pre-runs of the same config produce the same end block number with different hashes:

compute  build 31183512437 : end_block=24410463  end_hash=0xf8115f05…
stateful build 31133523115 : end_block=24410463  end_hash=0x4525339a…

Running the compute suite against stateful-promoted datadirs, the replay reported success while skipping everything, and then every benchmark payload was an orphan — reth stuck on forkchoice updated message when syncing, nethermind 0 passed of 244 attempted. Hours of wall-clock per client, and it reads as a client bug rather than a wrong datadir. With this change the run stops up front with the datadir is on a different chain than the bundle was recorded against.

Testing

  • TestVerifyPreRunBundleHeadFromFixturesArtifact — the regression: pre_runs with no local_fixtures_dir, bundle located via the source. Covers wrong-hash-now-rejected, matching-head-still-applied, and the no-locator/no-local-dir case. On master the first of these passes silently, because the early return means no verification happens at all.
  • TestPreRunBundleDir — table-driven over local-wins, artifact fallback, default subdir, and both nothing-to-resolve cases, plus a compile-time PreRunBundleLocator assertion.
  • TestReadPreRunBundleInfoAt — reads a bundle dir directly and asserts it agrees with the parent-form call.
  • All pre-existing TestVerifyPreRunBundleHead subtests still pass; the config-derived fallback is what keeps them green, and dropping it broke them, which is how I found it was load-bearing.

gofmt, go vet and go test -race are clean for pkg/executor, pkg/builder and pkg/runner. golangci-lint run --new-from-rev=origin/master reports 0 issues for pkg/executor and pkg/builder; it cannot load pkg/runner in my environment (pkg/podman needs cgo gpgme, which also fails on pristine master here), so pkg/runner was built, vetted and tested with -tags containers_image_openpgp instead — CI will cover it properly.

… artifact

verifyPreRunBundleHead guards the replay's by-number skip: a datadir at
the bundle's height but on a different chain would otherwise skip every
line and benchmark the wrong state. It only ran when pre_runs named a
local_fixtures_dir, so since #296 — which let a bundle be resolved out
of the extracted fixtures artifact instead — it silently no-opped for
exactly the configs that rely on that path.

Ask the source where its bundle is, via a new optional
PreRunBundleLocator, rather than re-deriving the path from config: the
bundle may sit at a local directory or inside the artifact, and only the
source knows which. EESTSource now resolves it once, for both the replay
and this check. The config-derived path stays as a fallback for callers
without a live source.

ReadPreRunBundleInfoAt reads a bundle directory named outright, which is
what a runner-side pre_runs source has; ReadPreRunBundleInfo keeps its
parent + PreRunBundleSubdir form.

An unlocatable bundle now warns instead of passing quietly, since the
skip it guards runs either way.
@skylenet
skylenet merged commit 2137e47 into master Aug 11, 2026
8 checks passed
@skylenet
skylenet deleted the fix/prerun-head-verify-artifact-bundle branch August 11, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pre-run replay: verify the datadir head hash, not just the block number, when the bundle comes from the fixtures artifact

1 participant